Skip to content

feat(backend): pluggable stem-separation backend (Demucs ↔ MSST/BS-RoFormer) - #141

Merged
slittycode merged 2 commits into
mainfrom
claude/keen-hopper-OQ1fV
Jun 3, 2026
Merged

feat(backend): pluggable stem-separation backend (Demucs ↔ MSST/BS-RoFormer)#141
slittycode merged 2 commits into
mainfrom
claude/keen-hopper-OQ1fV

Conversation

@slittycode

@slittycode slittycode commented Jun 3, 2026

Copy link
Copy Markdown
Owner

What & why

ASA's stem-separation stage was hard-wired to torchaudio Hybrid Demucs. This adds a pluggable separation backend so it can be swapped for the stronger MSST/BS-RoFormer models from SUC-DriverOld/MSST-WebUI, driving MSSeparator inference directly (no PySide6 GUI). The default stays Demucs and the stems JSON contract is unchanged (vocals/bass/drums/other, 44.1 kHz), so the stemAnalysis overlay, pitch/note translation, and MT3 all behave identically.

Selected with ASA_SEPARATION_BACKEND=msst (default demucs). Any failure degrades gracefully back to Demucs — this is a default-off experiment, mirroring loudness_backend.py.

Design: subprocess, not in-process import

MSST runs as a subprocess under its own venv (ASA_MSST_PYTHON) rather than an in-process import. This was the key architectural finding from a 4-agent design review, and it solves five problems at once:

  • Worker path works — server.py always invokes analyze.py under ./venv/bin/python, so an in-process MSST import could never see MSST's deps (it'd be dead code on the product path).
  • JSON contract stays clean — MSST's progress bars/logging are forced to stderr and captured by the subprocess, never touching analyze.py's stdout JSON (tripwire feat: value-first backport — genre profiles, MixDoctor, broken-pipe fixes #1).
  • Dependency isolation — MSST pins librosa==0.9.2, numpy<2, its own torch; they can't coexist with ASA's torch==2.10.0. The MSST venv is built from MSST-WebUI's own requirements.txt.
  • No namespace collision — MSST's top-level utils/inference packages don't shadow ASA's.
  • Clean resource lifecycle — del_cache() runs in the runner's finally.
analyze.py (product venv) → separate_stems_backend()
  ├─ demucs → analyze_audio_io.separate_stems()        [unchanged default]
  └─ msst   → subprocess: $ASA_MSST_PYTHON msst_separate_runner.py …
                └─ MSST venv: resample→44.1k, separate(), transpose, write canonical WAVs,
                   print one-line JSON manifest; any failure → fall back to Demucs

Changes

New

  • apps/backend/separation_backend.py — dispatcher + model registry (ASA_MSST_MODEL, default scnet_4stem) + subprocess driver
  • apps/backend/scripts/msst_separate_runner.py — runs in the MSST venv: resamples to 44.1 kHz, transposes channels-last→first, maps stems by name (SCNet emits [drums,bass,other,vocals]), stderr-only logging, JSON manifest
  • apps/backend/separation_ab.py + scripts/ab_separation_backends.py — research-only A/B harness: synthetic gain-aligned SI-SDR smoke-test (explicitly labelled plumbing, not a real-music quality ranking — RoFormer is trained on real spectra) + optional real-track reference-free proxies + fair runtime (warm-up, min-of-repeats, device recorded)
  • apps/backend/requirements-msst.txt — separate-venv setup pointer
  • apps/backend/tests/test_separation_backend.py — 16 tests (no MSST install needed)

Edited

  • apps/backend/analyze.py — route all three separation sites (--separate, --pitch-note-only, --mt3-only) through separate_stems_backend
  • CLAUDE.md, apps/backend/JSON_SCHEMA.md — env vars, core-files, scripts, dep-isolation; schema note that separation is backend-agnostic

Correctness notes (from the review)

  • MSST temp dirs reuse the sonic_analyzer_demucs_ prefix so cleanup_stems reclaims them (no leak).
  • Runner writes 44.1 kHz stems, matching the hardcoded analyze_loudness(stereo, sample_rate=44_100) in _run_per_stem_analyses — otherwise per-stem LUFS would be silently wrong.
  • The 2-stem bs_roformer_vocals registry entry is research/A-B-only (leaves bass/drums empty); the default scnet_4stem fills all four.

Verification

  • test_separation_backend — 16 passed
  • test_analyze — 147 passed (EXPECTED_TOP_LEVEL_KEYS unchanged)
  • test_phase1_golden — 11 passed (golden snapshot unchanged)
  • No top-level JSON key added; default path is a pure pass-through to Demucs.

MSST end-to-end requires an MSST-WebUI checkout + 4-stem checkpoint (see requirements-msst.txt); the subprocess boundary is mocked in tests so CI needs no MSST install.

https://claude.ai/code/session_01PCXgSHYKWnrD4PiRkwzJxR

Generated by Claude Code

@claude
feat(backend): pluggable stem-separation backend (Demucs ↔ MSST/BS-Ro…
ea96482 
…Former)

Add a selectable separation backend so the Demucs stems stage can be swapped
for the stronger MSST/BS-RoFormer models from SUC-DriverOld/MSST-WebUI, driving
MSSeparator inference directly (no PySide6 GUI). Default stays Demucs; the
stems JSON contract (vocals/bass/drums/other, 44.1 kHz) is unchanged.

- separation_backend.py: ASA_SEPARATION_BACKEND dispatcher + model registry,
  mirroring loudness_backend.py (env-select, graceful degrade-to-Demucs on any
  failure). MSST runs as a subprocess under its own venv (ASA_MSST_PYTHON) so it
  works on the worker path (server.py hardcodes ./venv), keeps MSST stdout off
  analyze.py's JSON contract, and isolates MSST's conflicting deps.
- scripts/msst_separate_runner.py: runs in the MSST venv; resamples to 44.1 kHz,
  transposes channels-last→first, maps stems by name (SCNet order varies), forces
  MSST logging to stderr, emits a one-line JSON manifest.
- analyze.py: route all three separation sites (--separate, --pitch-note-only,
  --mt3-only) through separate_stems_backend.
- separation_ab.py + scripts/ab_separation_backends.py: research-only A/B harness
  — synthetic gain-aligned SI-SDR smoke-test (labelled plumbing, not a quality
  ranking) plus optional real-track reference-free proxies and fair runtime.
- requirements-msst.txt: separate-venv setup pointer (MSST brings its own torch).
- tests/test_separation_backend.py: 16 tests (dispatch, fallback, registry,
  runner helpers) — no MSST install required.
- Docs: CLAUDE.md env vars + core-files + scripts + dep-isolation; JSON_SCHEMA.md
  notes the backend is selectable and the schema backend-agnostic.

Default path unchanged: test_analyze (147) and test_phase1_golden (11) green.

https://claude.ai/code/session_01PCXgSHYKWnrD4PiRkwzJxR
slittycode
slittycode commented Jun 3, 2026
View reviewed changes

@slittycode slittycode left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE

Summary

Adds a pluggable stem-separation backend (ASA_SEPARATION_BACKEND=msst) via subprocess isolation — the right architectural call given MSST's dep conflicts with the product venv. The default path is an untouched pass-through to Demucs; the stems JSON contract (vocals/bass/drums/other, 44.1 kHz PCM16) is unchanged. Frontend and WASM CI passed; Backend CI is still running (Essentia install is slow), but the PR description's test counts (16 new + 147 existing + 11 golden) are credible given the change doesn't touch any DSP logic.

Findings

Worth considering (neither blocking):

  1. No happy-path MSST dispatch test. DispatchTests covers: Demucs default, MSST missing env → fallback, subprocess nonzero exit → fallback. RunnerManifestParseTests covers manifest parsing on success. What's missing is a test exercising the full chain: separate_stems_backend → _separate_via_msst_subprocess → subprocess exits 0 with valid JSON → stems returned. The individual pieces are tested in isolation but the integration isn't. Low effort to add — mock subprocess to return exit 0 + a valid manifest JSON, mock os.path.isfile to return True, assert separate_stems_backend returns the expected stems dict.

  2. 30-minute subprocess timeout (separation_backend.py, subprocess.run(..., timeout=1800)). When MSST hangs rather than failing cleanly (e.g., stalled on GPU memory, blocked weight download), the analysis worker blocks for up to 30 minutes before TimeoutExpired propagates and triggers the Demucs fallback. Acceptable for a default-off experiment; worth knowing before enabling in production.

Test results

Frontend: pass. Loudness WASM: pass. Backend: in progress. PR-reported counts (16/16 test_separation_backend, 147/147 test_analyze, 11/11 test_phase1_golden) are consistent with the diff scope.

Phase boundary check

Clean. separation_backend.py and msst_separate_runner.py are Phase 1 plumbing only. No Phase 2/3 involvement. No Phase 1 schema fields added, removed, renamed, or retyped. stemAnalysis schema note correctly updated to "backend-agnostic". Tripwire #1 (stdout purity): contextlib.redirect_stdout(sys.stderr) wraps the entire MSST execution block in the runner; json.dump(manifest, sys.stdout) happens after the context exits. All three separate_stems call sites in analyze.py correctly migrated to separate_stems_backend.

Generated by Claude Code

@claude
test(backend): cover the MSST happy-path dispatch chain
08b4dec 
Add an integration test exercising the full success path:
separate_stems_backend -> _separate_via_msst_subprocess -> subprocess exits 0
with a valid manifest -> stems returned with no Demucs fallback. The pieces
were tested in isolation; this closes the end-to-end gap (PR #141 review).

https://claude.ai/code/session_01PCXgSHYKWnrD4PiRkwzJxR
@slittycode
slittycode merged commit 8476af1 into main Jun 3, 2026
3 checks passed
@slittycode
slittycode deleted the claude/keen-hopper-OQ1fV branch June 3, 2026 22:47
This was referenced Jun 4, 2026
Merged
Merged
slittycode added a commit that referenced this pull request Jun 5, 2026
@slittycode @claude
feat(backend): make MSST separation run for real + ground-truth SDR A…
533fbb9 
…/B + licence gate (#144)

Follows PR #141's pluggable separation backend. Confirms the licence position
(the campaign's stated prerequisite), makes the MSST runner work end-to-end, and
adds a ground-truth-stems SDR mode so Demucs-vs-MSST can be measured on real
isolated stems. Default path untouched: ASA_SEPARATION_BACKEND still defaults to
demucs; analyze.py's JSON contract is unchanged.

Licence gate (previously unconfirmed):
- MSST-WebUI code is AGPL-3.0; both registry checkpoints (scnet_4stem,
  bs_roformer_vocals) are CC-BY-NC-SA-4.0 (NonCommercial). That is a strictly
  worse posture than the (MUSDB-grey) Demucs incumbent, so the MSST backend is
  research / NonCommercial-only and NOT promotable to a commercial default --
  promotion is licence-gated, not quality-gated.
- Durable in-code guardrails: LICENCE GATE notes on _MSST_MODEL_REGISTRY and in
  requirements-msst.txt. Full map + rationale + A/B results:
  incorporations/msst-separation-licence-gate-2026-06-05.md

msst_separate_runner.py -- 3 integration fixes (never exercised before; PR #141
had no MSST install):
- chdir into the checkout (MSSeparator import reads data_backup/webui_config.json
  via a relative path); caller paths resolved to absolute first so they survive.
- drop the injected stdlib logger (MSSeparator relies on MSST's own get_logger
  with a console_handler attribute); its handler targets stderr and the existing
  redirect_stdout keeps the stdout JSON contract clean.
- sample-rate read: regex fallback for configs carrying !!python/tuple tags.

separation_ab.py -- ground-truth reference-set mode (--ref-dir): loads
MUSDB-style {mixture,vocals,bass,drums,other}.wav, runs each backend, scores true
per-stem SI-SDR + aggregate; warmup-skip for the subprocess MSST backend (it
reloads per call). New tests/test_separation_ab.py (loader + aggregation +
si_sdr). Existing separation tests stay green.

NonCommercial research A/B (5 MUSDB18 test tracks, 7s, CPU, mono gain-aligned
SI-SDR -- a preliminary proxy, NOT museval): Demucs 8.08 dB @1.3s vs scnet_4stem
6.22 dB @40.8s (~32x slower on CPU). MSST wins vocals (10.77 vs 8.62) but trails
overall; this contradicts published museval SDR, so the short-clip proxy is the
limiter -- a full-length re-run is the honest basis for any quality claim. The
gate conclusion is unchanged either way: do not promote.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
slittycode added a commit that referenced this pull request Jun 5, 2026
@slittycode @claude
docs: sync AGENTS.md files against post-#141 codebase (#142)
8ed8e4c 
* docs: sync AGENTS.md files against post-#141 codebase

Two files drifted when the separation backend landed in #141 (which updated
CLAUDE.md but not the per-app AGENTS.md overlays) and when patchSmith landed
in #130.

apps/backend/AGENTS.md:
- Add separation_backend.py entry (selectable Demucs ↔ MSST backend)
- Add separation_ab.py entry (research-only A/B harness)
- Add tests/test_separation_backend.py entry
- Add msst_separate_runner.py and ab_separation_backends.py to the
  Operator and Research Scripts section

apps/ui/AGENTS.md:
- Add patchSmith.ts entry (Phase 3 Vital preset generation service)

No code changes; CLAUDE.md was already complete after #141.

https://claude.ai/code/session_011v17uVcfV8rM9EQpRn72Ra

* docs: fix patchSmith.ts description in apps/ui/AGENTS.md

The initial entry incorrectly said the preset builder uses Phase 2
recommendations. patchSmith.ts imports only Phase1Result — every
parameter cites a Phase 1 field per PURPOSE.md invariant #2. Correct
the description to reflect that.

Caught by review on PR #142.

https://claude.ai/code/session_011v17uVcfV8rM9EQpRn72Ra

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@slittycode @claude